1. Field of the Invention
The present invention relates generally to file management. In particular, the present invention relates to file management in shared file systems.
2. Background of the Invention
Advances in disk storage technology allow file systems to manage large volumes of data, with increasing number of files stored in a single file system. Typically a file manager traverses the file system for selecting candidate files that may be of interest to a user. As users require operations such as backup to complete within an assigned window, often 8 to 24 hours, candidate file selection performance becomes a limiting factor on the number of files that can be managed by the file manager within a single file system. Therefore, administrators typically “split” collections of files into large numbers of smaller file systems, resulting in higher costs for provisioning and management.
Conventional techniques for candidate file selection have the file manager traversing the file system directory structure and performing a “stat” operation on every file and every directory. The “stat” operation involves retrieving the attributes of a file or directory necessary to determine if a file is a candidate. Typically, this is performed by reading the file “inode” which contains information about the file, such as size and the last time it was changed. The inode does not contain the file name. The file names are stored in a directory structure which includes file names and corresponding inode numbers. The stat is also necessary for locating sub-directories, which must also be traversed by the file manager to locate all files in the file system.
Performing candidate selection in such a manner requires a minimum of one stat operation for every file or directory. Typically, the directories impose an arbitrary order on the files, causing the stat operations to generate random small reads of file metadata. The file manager has no knowledge of the number files and/or subdirectories that reside beneath an arbitrary directory in the namespace.